Eyeing the Camera: into the Next Century

In the two centuries of photography, there has been a wealth of invention and innovation aimed at capturing a realistic and pleasing full-color two-dimensional representation of a scene. In this paper, we look back at the historical milestones of color photography and bring into focus a fascinating parallelism between the evolution of chemical based color imaging starting over a century ago, and the evolution of electronic photography which continues today. The second part of our paper is dedicated to a technical discussion of the new Foveon X3 multi-layer color image sensor; what could be descried as a new more advanced species of camera sensor technology. The X3 technology is compared to other competing sensor technologies; we compare spectral sensitivities using one of many possible figures of merit. Finally we show and describe how, like the human visual system, the Foveon X3 sensor has an inherent luminance-chrominance behavior which results in higher image quality using fewer image pixels. The past two centuries Color Sensing in Film and Digital Photography The history of color photography is rich with exciting progress in technologies for color capture and color reproduction. Examining this history, we find that in many ways the development of digital photography is following a path parallel to that of film photography, offset by about a century. The parallels extend back to black-and-white photography as well, but that takes us a bit off topic. Inspired by Hermann Helmholtz's recent revival of Thomas Young's tri-chromatic theory of human color perception, in 1860 James Clerk Maxwell clarified the details of primaries and the idea of a color triangle covering only a portion of all possible colors. In 1861, he applied these ideas in the first demonstration of three-shot color photography, shot through three color filters, and demonstrated additive color reproduction using three projectors. Similarly, initial efforts toward color electronic photography used separate exposures for each color, and additive reproduction, by adapting television systems to frame-sequential color. An early three-shot-color electronicstill-photography example was the 1966 Surveyor 1 spacecraft, which used a vidicon with RGB filter wheel to electronically capture color images from the surface of the moon. Three-shot cameras with glass plates were used around the turn of the century, for example by Sergei ProkudinGorskii, photographer to the Czar of Russia. Reproduction was done by additive projection, as Maxwell did, as well as by subtractive sandwiches, as demonstrated in 1869 by Louis Ducos du Hauron and Charles Cros. In the late twentieth century, we saw the development of three-shot digital cameras with solid-state sensors, which are still used for professional still-life work; both additive reproduction (on screen) and subtractive (on print) became common for digital work. Dr. Hermann Vogel’s accidental discovery of dye sensitization of emulsions in 1873 led to a great increase in the practical applicability of photography—originally impractical with only blue-sensitive films. Corresponding improvements in digital sensors were needed a hundred years later to extend mostly-red-sensitive CCD sensors into the blue end of the spectrum before they would be suitable for color photography, around 1973. Ducos du Hauron helped move the three-shot camera concept toward a one-shot camera, by working on optical beamsplitters to expose three plates at once. Frederic Ives developed the concept further, and made practical color cameras. Three-plate film cameras, such as the Devin TriColor, were used through the first half of the twentieth century, overlapping with other technologies. The Technicolor movie camera is a famous success story of that class. Though collapsing the color sensing into a single shot solved motion problems, it left the difficult alignment problem in the reproduction stage. Decades later, on the parallel digital path, prism-based digital color separation cameras, such as the Foveon, suffered a corresponding alignment difficulty in their manufacture, making them rather expensive. Ducos du Hauron also started another important technology track, of what has been called screen plates or mosaics, but it was John Joly who first made it work via his carefully ruled micro-strips of red, green, and blue ink. The striped color film was later modified into a random mosaic in the Autochrome process of the Lumiere brothers, around 1904, and further improved as Agfacolor film, with versions around 1912, 1916, and 1923. Correspondingly, color CCD imagers evolved from using striped filters to using improved mosaic patterns of filters, mostly converging in the 1990s on the Bayer pattern, introduced by Bryce Bayer in 1976. The integrated color filter array enabled single-plate and single-sensor color cameras, which led to a surge in popularity of color photography with these simplified devices. But the division of plate area or sensor area into tiny regions, each sensitive to only one-third of the visible spectrum, left a lot to be desired in sensitivity, clarity, color accuracy, and freedom from sampling artifacts. Many saw that the key next step would be a layered arrangement of color-sensitive planes. Kodachrome in 1935, Agfacolor Neu in 1936, and Polacolor film in 1957 were the culminations of several intense efforts to implement such an approach in film. Correspondingly, many groups have worked to find a way to make multi-layer solid-state color sensors, sometimes trying to use the “vertical color filter” inherent in a semi-transparent silicon substrate. The Foveon X3 three-layer silicon imager, announced in 2002, is the culmination of one such effort. Figure 2. The Sigma SD9 is the first digital camera to use a fullcolor multi-layer sensor technology: The Foveon X3 sensor. Wavelength-dependent absorption depth Figure 3 shows a schematic drawing of a sensor that absorbs first the blue wavelength photons as the incident light enters the device, then the green photons, and finally the red photons at the deepest layer. Three separate PN junctions are buried at different depths inside the silicon surface and used to separate the electron-hole pairs that are formed by this naturally occurring property of silicon. As expected, the depths of the electrodes are the key variables that determine the spectral sensitivities of such a device. Figure 1. The introduction this year of Foveon X3 technology achieves for solid-state sensors what Kodachrome did for color film in 1935. Of course, once such a breakthrough has been introduced and proven viable, a rapid development of improvements does inevitably follow. In each of seven decades of color film development, progress has been amazing. It is reasonable to expect similar progress for silicon sensors, though on a modern accelerated schedule. Figure 3. A schematic drawing of a sensor stack that captures all of the incident photons, filtering the color components by the wavelength-dependent absorption of silicon. Just as the development of Kodachrome and other multi-layered films left some room for continuation of older technologies, such as the striped filter array of instant Polacolor2 transparency film, the introduction of multi-layer silicon sensors, such as Foveon X3, will leave room for other approaches for many years to come. The wavelength-dependent absorption coefficient of silicon, and corresponding mean penetration depth, are plotted in Figure 4. Silicon’s indirect band-gap makes the material semi-transparent. As light enters the sensor, it is absorbed to produce electron-hole pairs in proportion to the absorption coefficient, yielding many more charge carriers for short wavelengths than for long wavelengths near the silicon surface; both the rate of absorption and the remaining photon density decrease exponentially as the light penetrates the silicon, leaving only red and IR light to penetrate beyond a few microns. Figure 5 plots the absorption as a function of depth, which is an exponential function of depth for any wavelength. The higher-energy photons interact more strongly, have a smaller space constant, and thus the exponential fall-off with depth is more rapid, as shown. Foveon X3 Technology As we start the 21 century, several groups are striving to do for digital photography what Kodachrome and AgfaColor did for film photography in the first part of the 20 century: produce a multi-layer silicon sensor. The first commercial product to use the Foveon X3 technology, the Sigma SD9 (figure 2), uses just such a layered silicon sensor fabricated on a standard CMOS (complementary metal-oxide semiconductor) processing line. 350 400 450 500 550 600 650 700 750 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 wavelength, nm qu an tu m e ffic ien cy Figure 6. Wavelength vs. quantum efficiency. Figure 4 Absorption coefficient and penetration depth in Silicon, vs. wavelength.. 350 400 450 500 550 600 650 700 750 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 wavelength, nm re lat ive re sp on se 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 depth, microns ab so rp tio n pe r un it de pt h 400 nm solid